Papers with open-source vision-language models
LATTE: Learning to Think with Vision Specialists (2025.emnlp-main)
Copied to clipboard
Zixian Ma, Jianguo Zhang, Zhiwei Liu, Jieyu Zhang, Juntao Tan, Manli Shu, Juan Carlos Niebles, Shelby Heinecke, Huan Wang, Caiming Xiong, Ranjay Krishna, Silvio Savarese
| Challenge: | Open-source vision-language models excel on simple question-answering tasks, but struggle with complex questions that require both perception and reasoning. |
| Approach: | They propose a family of vision-language models that have LeArned to Think wiTh vision spEcialists by offloading perception to state-of-the-art vision models. |
| Outcome: | The proposed model achieves 4-5% gains over baselines across 6 benchmarks covering both perception and reasoning abilities. |
InfiniBench: A Benchmark for Large Multi-Modal Models in Long-Form Movies and TV Shows (2025.emnlp-main)
Copied to clipboard
Kirolos Ataallah, Eslam Mohamed Bakr, Mahmoud Ahmed, Chenhui Gou, Khushbu Pahwa, Jian Ding, Mohamed Elhoseiny
| Challenge: | Existing benchmarks fail to test the full range of cognitive skills needed to process long-form videos . |
| Approach: | They propose a benchmark to evaluate models' ability to process long-form videos rigorously. |
| Outcome: | The benchmark measures the cognitive skills of models in understanding long-form videos . it offers the largest set of question-answer pairs for long video comprehension . |